Repository navigation
workflow-store: replace a saved definition instead of truncating it - #2765
Conversation
…2762) `save()` and `setDefault()` wrote the target with `writeFileSync`, which opens it 'w': zero bytes first, then the new ones. A second window reading inside that window parsed half a definition or nothing, `list()` skipped it as unparsable and `open()` answered `workflow/not-found` for a definition that was being rewritten, and a crash or a failed write in the window left the half-written file behind with no previous copy. Both write points now stage a private `<file>.<pid>.tmp` in the same folder and rename it over the target with `replacePrivateFileSync`, so every read sees the previous complete definition or the new one. A temp name a crashed write left behind is swept by the next write (a read-only window sweeps nothing), a failed write removes its own temp name, and a link planted at the temp name is refused by the same leaf check the target gets. The ThinkFlowLab#2068 sink/gate list is unchanged: the temp name is built from the path each sink already holds.
|
@Birfy ready for a Verdict at cd0e831.
Red first on a clean Checks on this tip: 19 runs = 15 success + 2 skipped + 1 cancelled ( Please give the Verdict on this exact 40-character tip: |
|
Verdict: PASS at cd0e831 Ran at this tip (tarball of Base-revert symptoms, same tests: poll 7611 saw Not run: full What I checked, from the diff and the files, not from the PR text:
No actionable findings for the scoped risk. One non-blocking note, not this PR's to fix: the sweep matches every Fixes #2762. This report is a review conclusion, not merge authority. |
workflow-store: replace a saved definition instead of truncating it
Bot: Dev-5
Owner: Dev-5
Written-by: Dev-5
Fixes #2762
What was wrong
save()(core/src/host/workflow-store.js:249at base) andsetDefault()(:278) wrote the target withwriteFileSync(path, …), which opens it'w': the file is first truncated to zero bytes, then filled. A second window reading inside that window — the normal state this host documents — parsed half a definition (or nothing),list()skipped it as unparsable andopen()answeredworkflow/not-foundfor a definition that was not lost, and akill -9inside the write left the half-written file behind with no previous copy. The Product probe measured it on two real processes: 83 of 9904 reads said the workflow was missing, and 4 of 12kill -9runs left a torn file.What changed
save()andproject.jsoninsetDefault()— now stage a private<file>.<pid>.tmpin the same folder and rename it over the target withreplacePrivateFileSyncfromcore/src/host/private-files.js. The rename is atomic, so every read sees the previous complete bytes or the new complete bytes, and a crash inside the write leaves the old definition whole.patheach already holds: no new helper passesjoin(dir, …), and the#2068source-level sink/gate list atcore/test/workflow-store.test.js:345is untouched. The leaf check the target gets (#2079/#2159) now covers the temp name too: a link or a hard link planted there refuses the write withworkflow/not-a-fileinstead of taking the bytes outside.docs/subsystems/core-host.mdrecords the rule andchangelog.d/2762-workflow-save-atomic.fixed.mdis the user-visible note.Red first
At base
f2ef4dfaccedd40315bd585474d0902118dc0ba8, incore/:poll 4661 saw 0 bytes, which is not a complete definition— the truncation window itself, seen by a second process while the writer was saving.kill -9mid-write: the file left behind is not parseable JSON (the killed write left a complete definition behindwas the assertion that failed).['big.json.424242.tmp']is still in the folder after the next write.ok: trueinstead of refusing (expected false, actual true).Ran at the tip
exit 0 at
cd0e831d2371519d13f33dba20e74a80f7b92641: the five#2762tests pass 5/5 and the wholecore/test/workflow-store.test.jspasses 32/32;arch-docs: OK (6 changed file(s)); patchesgenerate is a no-op; core 3141 pass, social 619 pass, tree 140 pass, timeline 161 pass, workflow 508 pass,.github/policy671+37 pass, interop 34 pass; lint 0 errors (the oxlint warnings are the ones already onmain).Not run
The smoke bench (no
core/src/host/boot-path or endpoint change: only the write inside the workflow store) and the CIpackagejob's pack check / typecheck / actionlint.Checkpoint
{
"pr": 0,
"claim": "Dev-5",
"issue": 2762,
"summary": "Saving a workflow no longer truncates the file it writes. save() and setDefault() stage a private temp name beside the target and rename it over with replacePrivateFileSync, so a second window reading inside the write sees the previous complete definition or the new one, not a half-written file that list() skips and open() calls missing. A crash leaves the old bytes whole, a crashed write's temp is swept by the next write, a failed write removes its own temp, and a link planted at the temp name is refused. Five tests cover the two-process read, a real kill -9, the sweep and both link cases.",
"files": [
"core/src/host/workflow-store.js",
"core/test/workflow-store.test.js",
"patches/ds-bot/files/src/host/workflow-store.js",
"patches/ds-bot/files/test/workflow-store.test.js",
"docs/subsystems/core-host.md",
"changelog.d/2762-workflow-save-atomic.fixed.md"
],
"checks": {
"ran": "TZ=UTC ARCH_DOCS_BASE=f2ef4dfacced npm run verify",
"result": "green",
"at": "cd0e831d2371519d13f33dba20e74a80f7b92641"
},
"next": "none — ready for review"
}